Closed-Form Inversion of Backpropagation Networks: Theory and Optimization Issues

We describe a closed-form technique for mapping the output of a trained backpropagation network int.o input activity space. The mapping is an inverse mapping in the sense that, when the image of the mapping in input activity space is propagat.ed forward through the normal network dynamics, it reproduces the output used to generate that image. When more than one such inverse mappings exist, our inverse ma.pping is special in that it has no projection onto the nullspace of the activation flow operator for the entire network. An important by-product of our calculation, when more than one invel'se mappings exist, is an orthogonal basis set of a significant portion of the activation flow operator nullspace. This basis set can be used to obtain an alternate inverse mapping that is optimized for a particular rea.l-world application.